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1 Introduction 

The expression "spin glass" was originally coined to describe some magnetic al- 
loys in which there was observed non-periodic "freezing" of the orientations of 
the magnetic moments (or "spins"), coupled with slow response and linear low- 
temperature heat capacity characteristic of conventional glasses. The attempt 
to understand the cooperative physics of such alloys has exposed many previ- 
ously unknown and unanticipated fundamental concepts and led to the devising 
of new analytical, experimental and computer simulational techniques. These 
have had major ramifications throughout the whole field of study of prob- 
lems involving assemblies of strongly interacting individual entities in which 
competitive forces yield complex cooperative behaviour. Such problems are 
ubiquitous, not only throughout the breadth of condensed-matter physics but 
also biology, evolution, organizational dynamics, hard-optimization, and envi- 
ronmental and social structures. In consequence the expression "spin glass" 
has now taken on a wider interpretation to refer to complex glassy behaviour 
arising from a combination of quenched disorder and competitive interactions 
or constraints, and to systems exhibiting such behaviour. 

In these lectures I shall introduce and partially overview the key concepts 
and main theoretical techniques for understanding and quantifying model spin 
glasses, with only limited discussion of experiments and of history and no 
attempt at completeness. 

Despite the caveat of the last sentence, a brief historical introduction to 
set the scene does seem appropriate □. The earliest experiments drawing at- 
tention to spin glasses as potentially interesting systems were performed on 
substitutional alloys of magnetic and non magnetic metals. Evidence for ran- 
dom freezing was provided by the combination of the observation beneath 
a characteristic temperature of Mossbauer line-splitting in zero applied field, 
indicating a local hyperfine field due to local freezing of the magnetic mo- 
ments, and the absence of any corresponding magnetic Bragg peak in neutron 
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diffraction, demonstrating that the freezing is not periodic. In fact, even ear- 
Her, measurements of the susceptibihty had shown a maximum at a similar 
temperature, again indicative of non-ferromagnetic freezing, but the observed 
peak was rounded, suggestive of sluggish response. The possibility of a true 
phase transition was highlighted later when a.c. susceptibility measurements 
in which static fields were kept very low showed sharp cuspsa. The attempt by 
Edwards and Anderson El to produce a theory of this transition was the spark 
for a theoretical revolution which still continues. 

Random freezing is certainly a novelty and, as noted above, it was the 
attempt to explain the occurrence of this feature which stimulated discover- 
ies of great subtlety. However, a second set of early experiments exposed the 
presence of another, even more remarkable feature. These experiments demon- 
strated severe preparation-dependence effects and considerable slowing-down 
of response to external perturbations beneath the same characteristic tempera- 
ture as that found in the susceptibility experiments. Although its significance 
was not appreciated until later, one of these striking preparation-dependent 
features was observed in a d.c. susceptibility experiment; the susceptibility 
obtained by cooling the system in the measurement field (FC: field cooling) 
yielded a higher value than that obtained by first cooling injzeio field and then 
applying the measurement field (ZFC: zero field cooling) □. Figure 1 shows 
a more recent demonstration U. Similarly dramatic preparation-dependence 
was observed in measurements of the remanent magnetization, which, if mea- 
sured by cooling in the field and then removing it (TMR: thermoremanent 
magnetization), is greater than the isothermal remanent magnetization (IRM) 
obtained by first cooling, then applying and finally removing the field (see fig. 
2). These observations demonstrate that in the new "phase" there are many 
metastable states whose relative free energies vary in different ways with exter- 
nal perturbations and which have significant (free) energy barriers impeding 
motion from one state to another. 



2 The key ingredients 

Spin glass behaviour seems to require two essential ingredients. These are 
quenched disorder and frustration ^ 

"Quenched disorder" refers to constrained disorder in the interactions be- 
tween the spins and/or their locations. The spin orientations themselves are 
variables, (i.e. not constrained), governed by the interactions, external fields 

"In fact, recent studies have shown that the disorder can be effectively self-generated and 
are leading to new insight into conventional glasses, but for the moment we shall ignore this 
subtlety. 
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Figure 1: D.c. susceptibility measurements^ for two CuMn alloys with 1.08 and 2.02 at % 
Mn. Curves (a) and (c) were obtained by cooling in the measurement field (FC),(b) and (d) 
axe the results of zero-field-cooled (ZFC) experiments; from Nagata et. al.^ 

and thermal fluctuations, free to order or not as their dynamics or thermo- 
dynamics tells them. The spin glass phase is an example of spontaneous co- 
operative freezing (or order) of the spin orientations in the presence of the 
constrained disorder of the interactions or spin locations. It is thus "order in 
the presence of disorder" . At a deeper level, in real solids the time scale for 
the ordering of the spin orientations is short but that of ordering of the atoms 
or interactions is very long. 

"Frustration" refers to conflicts between interactions, or other spin-ordering 
forces, such that not all can be obeyed simultaneously. 

These features are readily visualized in a simple model Hamiltonian ap- 
propriate to an idealization of magnetic interactions between atoms with well- 
defined local moments. 

H = -Y,J^3S^.S,, (1) 

m 

where the i,j label the magnetic moments/spins, the Si are the correspond- 
ing spin orientation vectors, and measures the "exchange" interaction be- 
tween the pair of spins (ij). The variables are the {Si}, while the { Jy} are 
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Figure 2: Remanent magnatization as measured in Au-0.5% Fe at 1.2 K. IRM denotes 
isothermal remanent magnetization, TRM denotes thermoremanent magnetization; from 
Tholence and Tournier^ 

quenched/ constrained. In conventional experimental spin glass systems, the 
spin locations 11% are randomly located (on a lattice in the case of the sub- 
stitutional alloys mentioned earlier) and Jy is a function of (il, — which 
oscillates in sign with separation. Frustration arises in that pairs of spins get 
different ordering instructions through the various paths which link % and j, 
cither directly or via intermediate spins. For theoretical studios it is usually 
more convenient to consider models in which there arc magnetic spins on all 
sites but the interactions Jy are randomly positive or negative (and quenched); 
that such models still capture; the essence is borne out by computer simulations 
which yield results qualitatively similar to those of real experiments. 

A natural corollary of this recognition of the key ingredients is that exper- 
imentally the behaviour should not be restricted to metallic systems, provided 
one has disorder and frustration, and indeed the effects have now been seen in 
several insulating alloys. The canonical insulating example is EuxSri-xS, in 
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which only Eu is magnetic and for which the nearest-neighbour interactions 
are ferromagnetic, next-nearest interactions are antiferromagnetic. 

The relevance of frustration is that it leads to degeneracy or multiplicity 
of compromises; for example if one has a set of Ising spins at the corners of 
a polygon with nearest-neighbour interactions randomly ± J, then one cannot 
satisfy all the bonds if an odd number of them are antiferromagnetic and the 
ground state is degenerate. In extensive systems frustration can have major 
consequences on cooperative ordering. For example, a triangular antiferromag- 
net remains paramagnetic at all temperatures. Other periodically frustrated 
systems do order, in a rich plethora of compromise phases, both commensurate 
and incommensurate with the underlying lattice, but with sufficient quenched 
disorder no periodic compromise is possible. 



3 Analytic theory: thermodynamics 

Let us now turn to an analysis of spin glassesi, taking as prototype the random- 
bond Ising model with Hamiltonian 

= - ^ JyCTiCTj; = ±1. (2) 



where the i label the spins and the Jij are drawn randomly from distributions 
P{Jij) which are the same for all equivalent pairs of spin locations. Initially, 
we shall consider the spins to be on a lattice of sites («, j). 

The possibility of random ordering without ferromagnetism can be hinted 
at by a straightforward extension of conventional mean-field theory. Allowing 
for the lack of spatial symmetry, but ignoring self/cavity field and thermody- 
namic fluctuation effects, a simple extension of a conventional approximation 
yields the set of self-consistent mean-field equations 

{a,) = tanh(^ {aj)/kT), (3) 



where the ( ) brackets indicate a (possibly symmetry-broken) thermodynamic 
average 

^ ^ {<T,}=±1 

Conventionally, within mean-field approximation, the critical temperature 
Tc for the onset of frozen order is given by the existence of a non-trivial solu- 
tion (((t) ^ 0) to the linearized mean- field equation. In the present case this 
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corresponds to 

3 

Averaging over sites and bonds and (without justification) ignoring corre- 
lations between those averages yields 

[{<y^)]^Y.y^3w^)ykT,^ (6) 

where the [ ] brackets denote an average over the bond or site disorder. For 
a symmetric exchange distribution [Jij] is zero and there is no non-trivial 
solution to at finite Tc- There is no ferromagnetic solution. An analogous 
consideration similarly eliminates any other periodic order in this case. If, 
however, is first squared on each side and then averaged, again ignoring 
correlations and using [Jij] = 0, there results 

[(^»)'] = E[4H('^.>']/(fc7^c)^ (7) 

i 

which has a non-trivial solution at a critical temperature given by kTc ~ 
Jfj)^^"^ with 'order parameter" [((7^)^]. Thus, beneath this temperature 
one has a "frozen-spin" state but without periodic order. Of course, even within 
mean-field theory this analysis is inadequate in detail, but, as we shall see 
below, sophisticated mean-field theory also yields a frozen-spin state without 
periodic order. 

3.1 Replica theory 

Much of the further progress in understanding and quantifying the spin glass 
problem has employed an artifice known as replica theory. This was introduced 
by Edwards and Anderson lI to aid in the analysis of physical averages over 
quenched disorder. 

We are generally interested in statistically representative quantities, rather 
than specific instances. It is therefore of interest to look at averages over 
specific disorder. Indeed, it is a traditional tenet that physical macroscopic 
measurements on nominally equivalent random systems are overwhelmingly 
dominated by their mean values; for example, the susceptibility per unit mass 
of one piece of CuMn alloy is expected to be the same as another of the 
same concentration, provided they are similarly prepared and subject to the 
same external fields, even though the precise locations of the individual atoms 
differ. Recently, we have come to realize that subtleties of macroscopic fluc- 
tuations are possible and we shall reconsider this assumption later. For the 
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present, however, we shall study the disorder-averaged system. It is, however, 
important that one averages physical observables if one is to obtain physically 
relevant results. Thus it is sensible to average the free energy F = —kTlnZ, 
where Z is the partition function 

Z ^Trexp{~PH), (8) 

but not to average Z itself. Unfortunately, whereas Z is relatively easy to 
average, being a sum of exponentials, InZ is much harder to average. The 
replica trick is an artifice to transform the hard average over In Z into an 
easier one over an effective Z. 

In particular, the replica trick starts with the mathematical identity 

InZ = lim -(Z" - 1). (9) 

Then Z" may be interpreted as the partition function of n identical replicas 
of the original system. Introducing a replica label a = l,...,n, Z" may be 
written as 

= Tr„ exp H'^/kT^ , (10) 

where is the Hamiltonian with dummy variables labelled by an extra index 
a and rr„ is the trace over all the variables. Because is a sum over 
exponentials it is relatively straightforward to average in terms of cumulants. 
For the random-bond Ising model one obtains 

[Z-] Tr^ exp <( }_^[J:,y {kTY ^ «af • • • )(a>f • • • af ) 

Q./3,... 

where [Jij]c indicates the rth cumulant moment of Jy and a,P, ...5^ indicate 
r replica labels, each taking values from 1 to n. ^ 

At this point it is convenient to follow Edwards and Andersonl3(EA) and 
Sherrington and Southern lI(SS) and restrict P[J) to a Gaussian form, since 
then all the cumulant moments higher than r = 2 vanish. To further restrict 
the problem to single parameters to characterize each of ferromagnetic and spin 
glass tendencies, the range (zj) can be restricted either to nearest neighbours 
(as EA and SS) or to all spins (Sherrington-KirkpatrickET). Then 




[F] ^ -kT\\m- { Trn exp 

n— >o n 



{ij) a a,P 



(12) 
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where Jo is the mean and J the standard deviation of the P{Jij). Thus we have 
replaced the original disordered system of eq. (3.1) by one with a temperature- 
dependent effective Hamiltonian 

(n n \ 

Jo E a^af + pP ^>^^>J (13) 

with no disorder but involving higher-dimensional spins with more complicated 
interactions and requiring analysis in the limit n 0. 



3.2 Replica mean- field theory 

For short-range interactions the expression in (^2|) cannot be evaluated ex- 
actly. By analogy with conventional magnetism, it is natural to consider first 
a mean-field approximation in which an interacting problem is replaced by 
an effective non-interacting system with self-consistently determined "fields" . 
For infinite-ranged systems with appropriate scaling of Jo^J with the num- 
ber of spins N such an analysis can be performed exactly, and in consequence 
the SK model and its variants have become the defining mean field models. 
These infinite-ranged models are usually treated by a procedure of mapping 
to macroscopic variables with extremally dominated generating functionals, 
but for orientational purposes I shall first follow a route closer to conventional 
mean-field theory. Thus one replaces 

E ^ E(^"< - <rny2)- ml = (aDn, (14) 

as) 

where ( )„ refers to a thermodynamic average against the effective Hamilto- 
nian and the order parameters mf , q'l'^^^ are to be determined self-consistently. 
Since eq. (13) is translationally invariant, we can assume a similar property 
for mf, g,^"''^ ^ 

The evaluation of [F] now becomes a single-site problem: 

[F] = - NkT lim — {rr„exp[/3JoZ V(cr"m" - (m")V2) 

a 

''provided Jo is non-negative; if Jo were negative we would need to allow for antiferromagnctic 
order instead. 



8 



+ if3jfz{n + 2 ^ (a"tT^g("« - q^"'^^^/2))] - 1}, (16) 

(a/3) 

where z is the coordination number, the trace is now single-site, and m" and 
qiai3) g^j,g given by 

^„ ^ rr„a"exp(-/3g„) ^ rr„a"a^ exp(-/3g„) ^^^^ 

Tr„exp(-/3i?„) ' Tr™ exp(-/3/f„) 



where —f3Hn is the argument of the exponential in (p_6D, or, equivalently, by 
the extremal equations 

where 

F„ = -fcrinrr„exp(-/37?„). (19) 
3.3 Replica- symmetric Ansatz 

Equations ( p^ ) represent a set of self-consistent equations. Were n a fixed finite 
integer, they would be straightforward to solveo but with the limiting proce- 
dure n — !■ the problem is more difficult, requiring an appropriate analytic 
continuation. For this reason a further simplifying Ansatz was proposed'^, the 
so-called replica-symmetric (RS) Ansatz in which one assumes 

m" = m, ah a, 9^"'^) = q, ah a ^ /3. (20) 

This Ansatz is natural, since the replicas are mathematical artifices and are 
apparently indistinguishable. In fact, however, the situation is more subtle, 
but we defer further discussion of the subtlety until later. 

The computational advantage of the replica-symmetric Ansatz is two-fold; 
firstly, it permits ( p^ to be expressed in terms of independent spin terms, which 
may be evaluated readily, and secondly, it provides for straightforward analytic 
continuation. To see the first of these properties, note that the "interacting" 
term in the exponent 

2^aVV"'') ^ g{(^a")2-l} (21) 

a/3 a 

which can be reduced to single-cr form via the identity 

exp(Aix2) = (2^)-i/2 / dyexp(-yV2 + {2iiY'^xy)- (22) 
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here we take fi = {pj^zq and x — X)"'"- The shuple analytic continuation 
occurs because now all cr" enter linearly with the same coefficient. Evaluation 
of the trace and taking of the limit n ^ yields 

[F] ^N\^^-^{l-qf -kT J dhP{h) lii{2 cosh (3h)Y (23) 
where Jq = Jqz, J — Jz^^^ and m and q are determined self-consistently from 
m= I dhP{h)t&nh(3h, q^ I dhP{h){t'Anhphf, (24) 



with 

P{h) = (27rJg2)-i/2 exp(-(/i - J^7nf/2Jq^). (25) 
Within this approximation ni and q can also be identified as 

m=[(a,;)], q=[{<y^)% (26) 

i.e. as the average magnetization and mean-square disorder-averaged local 
moment IjI^. Hence non-zero q implies a cooperatively frozen magnetic state, 
while non-zero m implies that that frozen state has a ferromagnetic component. 



Thus, solutions to eqs. (24) and ( p5| ) fall into three possible categories: 

(a) paramagnetic if both m and q are zero. 

(b) ferromagnetic if both m and q are non-zero. 

(c) spin glass if q is non-zero but m is zero. 

The resultant phase diagram is shown in fig. 3a, while fig 3b shows for 
comparison a typical experimental phase diagram. When account is taken of 
the different scalings with conccntratioiippf effective interactions Jqi J there is 
a qualitative accord between the figures Q. 

Let us concentrate now on the spin glass phase. As the temperature is 
lowered beneath the critical temperature Tg — J /k, q grows continuously from 

'^This identification (and extensions of it) follows from writing {(TiY as the trace of ctJ...(t^ 
over r replicas of the disordered system 

, _ Trra] ■ ■ ■ < exp(~/3(gi + ■ ■ ■ + H^)) 
rr.exp(-/3(Hi + ■■■+//'■)) 

multiplying numerator and demoninator by Z"^*", taking the limit n — » and averaging 
over the disorder to yield 

[{^iY] = lim (o-°'^"V°"')„;ai ^ 0.2 + ■ ■ ■ + ar. 

n—*0 
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zero as 

q = T+lT^+0{r'y, T={Tg-T)/Tg. (27) 

This yields a cusp in the zero-field susceptibility x{T), as is observed ex- 
perimentally. This is most readily seen from the fluctuation correlation form 
for the differential susceptibility: 

X(T) = {kT)-'N-' - (28) 

ij 

For the case Jq = 0, the terms in the summand are zero on average unless 
i = j and 

X{T) = xo{T) = (NkT)-' ^(1 - {aif), (29) 

i 

or passing to the average 

Xo(T) = {kT)-\l - [{a,f]) = {kT)-\l - q). (30) 
Hence, in the vicinity oiT — Tg one has 

Xo{T) = {kTg)-'{l-\r\+0{T^)), T>Tg, (31) 
(fcT,)-i(l-ir^+0(r3)), T<T,. (32) 

For Jo ^ 0, X has a similar cusp but with an overall enhancement factor 
giving 

x(T) = xo(T)/(l- Joxo(r)). (33) 
11 




Figure 4: (a) DifTerential susceptibility of a random-bond spin glass as given by replica- 
symmetric mean-field theory. Solid curves are for zero field, dotted curves for b = O.IJ. 
Curves (a) are for Jo/J = 0, curves (b) are for Jo/ J = 0.5; Sherrington and Kirkpatrick*. 
(b) External field dependence of a.c. susceptibility of Au Fe; Cannella and Mydosh^. 



In the presence of a finite external field b, P{h) of eq. ( P5[ ) is modified 
by replacing h h — b and x(T') is rounded, as shown in fig. 4a. A similar 
rounding of the cusp in x is found experimentally (fig. 4b); indeed it is the 
great sensitivity to such rounding which delayed the experimental discovery of 
a sharp transition. 

For continuous phase transitions one is used to looking for a diverging 
response function. Clearly, the uniform susceptibility is not such a response 
function for a spin glass transition. Rather, the divergent susceptibilities are 
the g-response to a random field and the related non-linear uniform suscep- 
tibility. The first of these, sometimes called the spin glass susceptibility, is 
defined as 

XSG = d^Q/db\ (34) 

where 

Q = Y.'^<y^)l ; (35) 

here ( refers to a thermodynamic average against a system with an applied 
field {hi\ = {b^i\, where the S^i = ±1 are random and quenched.. Formally 
performing the differentiation of (|^ yields the fiuctuation formll3 

XSG - ^((a.a,) - (a.)(a,))^ (36) 

For Jo = and for T > Tg one obtains within mean-field theory xsG = 
(T^ — Tg)^^, which diverges as T Tg. For a finite uniform b, higher-order 
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Figure 5: De Almeida-Thouless surface (indicated by chain-hatching) for the hmit of stabihty 
of the rephca-symmetric Ansatz for mean-field theory for a randon-bond Ising model. The 
Ansatz is unstable on the side of the surface closer to the origin. The phase line beneath the 
AT surface is calculated within the Parisi Ansatz. 



susceptibilities can be defined by 



m = N 



{(J^) =xb + Xsb^ + X5b^ + 



(37) 



Within the present mean-field theory, the non- linear suscfiplibilities xa, X5 etc. 
also diverge at T^: in fact, xs a-nd XSG a-i'e linearly relatecESEj. A similar Hivpr- 
gence of these higher-order susceptibilities has been observed experimentalljnJIlj, 
albeit with a critical exponent different from that given by mean-field theory. 

In fact, however, the replica-symctric Ansatz is not everywhere stable, 
even within mean- field theory The problem is clearly exposed by looking at 
the normal modes of fluctuation of q"^ and m" in replica space about their 
replica-symmetric values. Thus one introduces fluctuation terms 77, e by 



a/3 



a/3 



(38) 



where q, m are the replica-symmetric values, expands the free energy expression 
[F] to second order in the fluctuations and studies its normal-mode spectrum 
in the limit n — > 0. Beneath a surface in {T/ J, Jq/ J,b/ J) space, shown in 
fig. 5 and including all of the spin glass phase, the replica-symmetric Ansatz 
is found to be unstable against the replica-symmetry breaking ?7-modes. This 
instability surface is known, after its discoverers, as the de Almeida-Thouless 
(AT) surface, its projections as AT lines. 

The inadequacy of the replica-symmetric Ansatz shows in other ways. The 
first of these to be recognized is its prediction of an unphysical (negative) form 
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for the zero-temperature entropy of the random-bond Ising model B. Another 
is the result of an evaluation of the spin glass susceptibility XSG beneath Tg, 

a king account of fluctuations to harmonic order, which yields a negative value 
O, despite the manifestly positive-definite form of (|36|). 
Beneath the AT surface a more subtle Ansatz is required for a stable so- 
lution to the mean- field equations (p^),(p7|). Various schemes were proposed, 
whose analysis suggested that many levelSpOf symmetry breaking were neces- 
sary to achieve stability. One due to ParisiE3 has proven to satisfy all stability 
tests to date. 



3.4 Replica symmetry breaking: Parisi's Ansatz 

In general, the Parisi scheme involved an infinite hierarchy of replica subdi- 
visions It may be formulated as follows. Consider q"^ as a symmetric 
n X n matrix with zeros on its diagonals and taking the value q"^ — q^'^P^ 
on the off-diagonals The replica-symmetric Ansatz corresponds to taking 
all the off-diagonal elements to be identical, equal to q. Parisi's method may 
be viewed as a series of subdivisions. First the {n x n) matrix is subdivided 
into (toi X mi) blocks and all the elements of the off-diagonal blocks are given 
the value of qi. The (mi x mi) blocks on the diagonal of the {n x n) ma- 
trix are subdivided further into (to2 x 7712) sub-blocks and all the elements of 
the off-diagonal subblocks given the value of q2- The (to2 x 7712) sub-blocks 
on the diagonal are subdivided further, into (7773 x 7713) sub-sub-blocks. The 
elements of the off-diagonal (7773 x 7773) sub-sub-blocks are given the value (73, 
while the diagonal sub-sub-blocks are further divided, and so on, giving an 
infinite regression of subdivisions of the diagonal blocks, with 

n > mi > 77i2 > • • • > 1. (39) 



So far in this discussion of subdivision, we have been envisaging 77, 777^ as 
integers, but the next step in the analysis is to consider analytic continuation 



to 77 ^ 0, and with it the inversion of the order of (39), 

< 7771 < 7772 < • • • < 1, (40) 

together with relabellings with 

mk/mk+i ^ I - dx/x, qk-^q{x), Q<x<\. (41) 



''in fact, some models of current interest have only a single level of sub-division, but the model 
of Edwards-Anderson/Sherrington-Kirkpatrick requires the full hierarchical structure. 
"^Recall that the notation (a/3) refers to pairs of different labels af}. 
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Figure 6: Parisi order function q{x) for the random-bond Ising model in zero external field 
at small reduced temperature r. To leading order in t, qi = xi = t 



Finally, q{x) is treated as a variational-parameter function and the ex- 
tremum of [F] with respect to q{x) is taken. 

Within the region of parameters for which replica symmetry is stable, q(x) 
is a constant for all x, but in the region for which the RS-Ansatz is unstable 
q{x) is not flat. For small reduced temperature r = {Tg — T)/Tg the leading 
behaviour of q{x) is shown in fig. 6. 

Before turning to a discussion of the implications of the Parisi Ansai^ for 
observables such as the susceptibility, let us consider its interpretation E£l. To 
this end, it is useful first to introduce another concept, that of overlap, and of 
its distribution. 

The overlap between two microscopic Ising states s, s' is defined by 

g-' (42) 

i 

where {cr|} is the spin configuration in microstate s. The distribution of mi- 
crostate overlaps is given by 

P((z)-^p,p,,<5((?-r'), (43) 



where ps is the probability of the microstate s. In a conventional thermody- 
namic ensemble 

^ exp(-g./fcr) 

Trcxpi-Hs/kry ^ ' 

where if, is the value of the Hamiltonian in state s. 
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The application of the rephca procedure to the evaluation of [^'(g)], to- 
gether with mean- field theory using the Parisi Ansatz, yields EHl 

[P{q)] ^ f S{q~ q{x))dx = dx/dq. (45) 
Jq 

Equation (3.53) thus provides an interpretation of q{x) as the inverse of the 
average microstate distribution. 

Similarly one may define the distribution of overlaps between thermo- 
dynamic macrostates. The overlap distribution between two thermodynamic 
macrostates S, S' is defined by 

^ss' ^ j^^i mfmf , (46) 

i 

where {mf} is the thermodynamic average of {cr,;} in macrostate S. The 
macrostate overlap distribution is given by 

P{q)^Y.PsPs'S{q-q'^''), (47) 
SS' 

where Ps is the probability of macrostate S, 

p. = ^°"Tf /In 

}2s' exp{-Fs'/kT) 

with Fs the free energy of state S. 

An important conceptual link arises from the fact the P{q) can be shown 
to be identical to P{q). Hence, 

[Piq)] = dx/dq (49) 

and (dx/dq) is interpreted as giving the average overlap distribution of ther- 
modynamic macrostates in a Boltzmann-Gibbs ensemble. 

A conventional Ising ferromagnet has only two thermodynamic states, spin 
up and spin down, and 

Piq) = l{5{q-m^) + 5{q + m^)), (50) 

where m is the magnetization per spin. An infinitesimal field suffices to elimi- 
nate the peak at q — —m?. Correspondingly, q{x) is a constant at q = m? for 
all X. 

16 



Similarly, in a region of replica symmetry, in which q{x) is constant as a 
function of x, say with q(x) = qo, [P{q)] has a single delta function peak, at q — 
qo, which we interpret as implying a single thermodynamic state. By contrast, 
in the region of replica-symmetry breaking, in which q{x) has structure, [^'(q)] 
has weight over a range of q, indicating the existence of many thermodynamic 
states. For example, for T just smaller than Tg and in the absence of a magnetic 
field, where q{x) has the form shown in fig. 6, [^'(q')] has a delta function at 
q = q{l), arising from the plateau in q{x), together with a continuum between 
q = and q — q{l), arising from the ramp in q{x). The delta function is 
interpreted as the overlap of thermodynamic states with themselves (5 = S" in 
(^6|)), while the continuum corresponds to a range of overlaps between different 
non-equivalent macrostates. 

This identification of P{q) as an overlap distribution of thermodynamic 
states leads to the expression of thermodynamic observables as integrals over 
q{x). For example, we recall ( |29| ) for the zero-field susceptibility of a system 
with symmetric P{Jij): 

x(i)-(fcT)-i(l-[(cT,>2]). (51) 
Averages such as [(cti)^] follow directly from [P((z)], from relations such as 



[(^1^2 • • • <Jkf] ^ Jdq q^'iPiq)] - ^ dxq{x)'' . 



(52) 



Hence, in particular, 

[{ai)^] ^ I dq q[P{q)] ^j^dxq{x), (53) 

so that ^ 

X{T) = (fcr)-i (^1 - ^ q{x)dx^ . (54) 

For T > Tg,q{x) is zero and ( ^4[ ) reproduces the Curie law. For T < Tg, 
Parisi theory gives 

1- / q{x)dx = T/Tg (55) 

and 

X={kTgr\ <Tg. (56) 

This result of Curie-like behaviour above Tg and flat x{T) beneath Tg will 
be seen to match the experimental field-cooled results, as shown in fig. 1. 
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Figure 7; q(x) for a random-bond Ising model in Parisi mean-field theory for (a) T > Tat, 
(b) T < Tax- The hatched area determines the anomaly A = Xfc ~ XZFC- 



On the other hand if q{x) is replaced by q{\) in (^4|), the resultant x{T) is 
similar to the replica-symmetric result, with x(^) decreasing monotonically 
beneath Tg. This is reminiscent of the zero-field cooled or a.c. susceptibility 
measurements. Thus one has apparent correspondences between field cooling 
and the exploration of all thermodynamic states, and between zero-field cooling 
and trapping in a single thermodynamic state |^. 

It is interesting to note that the onset of replica-symmetry breaking rep- 
resents an unusual type of phase transition which can lead to discontinuities 
in observables even in the presence of a finite conjugate field. For example, in 
an external field q{x) is non-zero for all x, but as long as the temperature is 
above that of the AT instability it is not x-dependent. As the temperature is 
reduced through the AT value, q{x) acquires structure and the Gibbs average 
and single-state (or field-cooled and zero-field-cooled) susceptibilities start to 
diverge, despite the fact that the discontinuity in the former in the zero-field 
limit is removed in a finite field. The evolution of q{x) with temperature in a 
finite field H is illustrated in fig. 7, while the resultant susceptibility is shown 
in fig. 8. In fig. 7 curve (a) corresponds to a value of the reduced temperature 
T which is less than a critical value aH^^^ (where a = [3/(4 J^)]-'^/'^) and yields 

q{.x)^Ta^\{r+{T^ + 2HY'^ + ■■■). (57) 

As the temperature is lowered Ta increases until it reaches afP'^^ . Beyond this 
temperature g(0) is pinned at alP'^^ but g(l) continued to rise, as shown in 
curve (b). The difference between the two susceptibilities, Gibbs (or FC) and 

■'^More strictly one needs to consider dynamics, which is the subject of section 4 
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Figure 8: Susceptibility of an Ising spin glass in an applied field H, as predicted by Parisi's 
mean-field theory. The upper curve shows the full Gibbs average, obtained from the full 
q{x) and interpreted as the field-cooled (FC) susceptibility. The lower curve shows the 
result of restricting to one thermodynamic state, as obtained from 5(1) and interpreted as 
the zero-field-cooled susceptibility. 



single-state (or ZFC), is given by 



A=(fcT)-i (9(1)-^ li^)dx^ (58) 

and is indicated (without the [kT]^^ factor) by the hatched region of Fig. 7. 
The susceptibility curves of fig. 8 follow directly from ^ 

XFC = (kT)-^ (^1 - q{x)dx^ , (59) 

and 

XZFC = ikT)-Hl-q{l)). (60) 

The Parisi solution exhibits several other interesting featuies. In particular 
we shall mention ultrametricitycj'Ej and non-self-averagingc^. 

fFor convenience we use the labels FC, ZFC, although strictly the quantities given are the 
Gibbs ensemble and single-state susceptibilities. 
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3. 5 Ultrametricity 

A space is described as "ultrametric" if distances in that space obey the fol- 
lowing property: given three points a, b, c separated by distances dab, dba dca 
and labelled such that 

dab > dbc > dca, (61) 

then 

dab = dbc; (62) 

i.e. the two largest distances between three points are equal. To relate this 
to the present study, we note that distance and overlap are complementary; 
distance is a measure of separation, overlap of similarity. Thus a space of 
overlaps is ultrametric if for three states S, S", S" labelled so that 

^SS' < ^S'S" < ^S"S^ (gg-) 

one has 

g""' (64) 

i.e. the two smallest overlaps are equal. 

With the definition of overlap given in (46), Parisi theory predicts such 
ultrametricity. A particular interest in this result lies in its implication of 
hierarchical order. This implication is obvious if one considers a hierarchical 
tree as shown in fig. 9, takes its endpoints as the states and their pairwise 
overlap as determined by how far back in the evolutionary tree one needs to go 
to find a common ancestor, the overlap being smaller the farther back one needs 
to go. One readily sees that for any three states the two smallest overlaps are 
equal. This observation of ultrametricity underpins |^ the conception of spin 
glasses as having free-energy surfaces with "hills-and-valleys" structure, since 
if one considers valleys in a mountain landscape as "states" and the height of 
the lowest col between two valleys as their "separation" , then those separations 
are ultrametrically ordered. 

3.6 Non-self-averaging 

As discussed earlier, intuition from ergodic-type ideas in conventional statis- 
tical mechanics leads one to expect that physical observables will possess the 
self-averaging feature; i.e. for an observable A of positive expectation value 



N 



1 \A] 

''This picture was originally conceived as a result of more philosophical considerations and 
via computer simulations e.g. Kirkpatrick and SherringtonEJ 
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Figure 9: An evolutionary tree, illustrating the occurrence of ultrametricity. If the overlap 
between two final states is measured by the degree of evolution of their nearest common 
ancestor, as shown, then for any group of three final states, the two smallest overlaps are 
equal. 

The Parisi Ansatz indeed maintains self-averaging for the normal observables 
such as energy, free energy and their derivatives. On the other hand, however, 
the distribution of overlaps is not self-averaging and therefore neither are its 
moments. For example, consider A = q = N^^ Tlii^i)'^ ■ Then 

{^qf^[q^]-[qY = j\\x)dx-(^j\{x)dx^ , (66) 

which is zero if q{x) is flat (replica-symmetric) but not if q{x) has structure; i.e. 
replica-symmetry breaking leads to non-self- averaging of the overlaps, again a 
manifestation of the many non-equivalent states. In fact, within the Parisi 
Ansatz the overlap probability measure which is self-averaging is the distribu- 
tion function of the distribution function P(P(g)) E3. 

The full Parisi analysis of the SK model leads to a further deduction that 
the free-energy landscape evolves chaotically as the parameters are changed; 
i.e. a small change in the actual set of {Jij} or of the temperature or applied 
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field causes a non-trivial change in the hill-valley structure 



3. 7 The Sherrington- Kirkpatrick model 

Although mean-field theory can only be considered approximate for systems 
with short-range interactions, the above analysis is believed to be exact for 
a special model in which every spin interacts with every other spin, but via 
quenched random interactions with an. appropriate scaling with N. This is the 
Sherrington-Kirkpatrick (SK) model 19 characterized in its Ising version by a 
Hamiltonian. 

H = -'^JijUiCTj, (67) 

where the sum is over all pairs {ij) and the Jij are quenched parameters chosen 
randomly and independently from a distribution with moments 

[J^^ = Jo = Jo/A^, [J-,] = J' = JVN. (68) 

( p^ ) is exact for this problem with Jqz Jq, Jz^^^ — > J. This is readily seen 
by the following procedure: (1) replica theory is used to express [F] as in (12) 
but with (ij) now running over all pairs of sites, (2) the a— summations in the 
exponent are written as complete squares 

E<^" = ^((E^")'-1), (69) 

E->f->'-^((E-"-f)^-i), (70) 

iv) 

and eq. (3.25) used to express [F] in terms of integrals of effective single-site 
paramagnets over auxiliary fields as 



\F] = —kT lim lim — 



n„dm"n(„^)dg("« exp(-7V/35({m"}, {g^"'^)})) - 1 

(71)" 
where 

5({m"},{g("'^)}) = (m")V2 + {q^°'^^f/2-kT In Tr cxp[/3 Jq ^ a"m" 

+ {PJf{n + 2^a'^<j'^q'^"^^^)]. (72) 

(a/3) 



*This is not the case for all spin glass models. 
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Because g is independent of N , the integral of eq.([71[) is overwhelmingly domi- 
nated by the values of {m"}, {q^"'^-'} for which g is minimized. This extremum 
has 

where m",g^"''^ are as given in (p7|). The free energy [F\ is as given in (p^. 
Thus the further analysis of (|l^) discussed earlier is believed to apply to the SK 
model; mean field theory is exact in the limit N ^ oo. Computer simulations 
of P(gl.aiid of ultrametricity confirm the results obtained analytically for this 
model C 



3.8 Other infinite- ranged models 

Other infinite-ranged models are soluble by similar techniques, together with 
some further transformations, and are taken to define the corresponding mean 
field solutions. The general procedure is as follows 

(i) The partition function is given by 

Z^Tr^S}^M-m{S}) (74) 
where the S arc the microscopic variables. 

(ii) Replicas are used to express In Z in a form convenient to average over 
the quenched disorder 

InZ = lim -(Z" - 1); (75) 

rwO n 



OL 

(iii) An average is performed over the quenched disorder to yield 

[Z"] = Tr^s"} exp(-/3i?e//({S})) (77) 



(iv) For infinite ranged problems i?e// will always involve the microscopic 
variables only as functions of sums over all sites; i.e. in combinations 
E» S'^.N-^ '£^ S?Sf. This is the source of solvabihty. Relevance 
and physical meaningfulness also require appropriate A^— scaling of the 
quenched parameters in H, such that the resultant effective energy is 
extensive. 
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(v) Macroscopic variables are introduced via the insertion of appropriate 
representations of unity, for example 

•' i 

i 

These are then used to re- write Hef / in terms of macroscopic variables 
alone. 

(vi) The delta functions are re-written in exponential form; e.g. 

i •' i 

The trace over the microvariables now separates so as to yield formally 

[Z"] = J nadx°'Uapdy'''^exp{-N<i>{{x°',y"'^})) (80) 

where a;, y denote all the introduced macrovariables and <i> involves only 
a single-site microvariable trace. Since the number of macrovariables is 
finite the integral is dominated in the limit of large N by the extremum 
of This then yields a set of coupled equations for the extremal x,y, 
which are themselves identifiable as thermal averages; for example the 
extremal m" is identified with J^ii'^?)- 

(vii) There still remains the problem of the explicit evaluation of and of 
taking the limit n — s- 0. For this the Parisi ansatz is used (or a simpler 
ansatz if stable). 

In the above I have written the macrovariables S in a notation reminis- 
cent of spins and indeed there are many examples in which they are spins 
or analogues thereof (for example, quadrupoles or Potts spins [] or states of 
neurons q|) , but there are also examples in which the microvariables are more 
reminiscent of exchange interactions, with the disorder in the consequential 
spin ordering. An example of the latter situatinn occurs in the optimization of 
neural networks for associative memory recall E£l; we return to this later. 

■'For a review of the situation for vector and Potts spin glasses, but excluding consideration 
of a recently recognised subtlety foir-Potts dimension greater than four, the reader is referred 
to an earlier review by the author EZI i— i 
*A discussion of applications to neural networks is given in section 5; see also refcj. 
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3.9 One step replica symmetry breaking 

As noted earlier, the SK model requires the full hierarchy of replica symmetry 
breakings, implying a complex free energy landscape with hills and valleys on 
all scales. Recently, however, interest has turned to some systems which require 
only a single step of replica symmetry breaking (IRSB) but which turn out 
to have other subtleties and probable relevance to conventional glasses. The 
full picture of these systems is still being developed but one of the important 
ingredients in the emerging understanding concerns the character of the IRSB. 
In IRSB the Parisi order function q{x) has the form 

q{x) — Qo; < X < m 

qi;m< X <1 ;gi > go- (81) 

whereas in the RS phase q{x) = q; all x. 

There are two possible scenarios for the onset of IRSB; in 'continuous 
IRSB' (qi — qo) grows continuouly as the threshold is passed, while in 'discon- 
tinuous IRSB' the onset of a non-zero {qi —qn) is discontinuous. In both cases 
the onset of the anomaly A = (gi — go)(l ~ ™) is continuous, so discontinuous 
IRSB commences with a continuous growth of (1 — to). 

In terms of the overlap distribution [P(q)] the picture is as follows. Above 
the critical temperature (or other control parameter), in the RS region, [/*(<?)] 
has a single delta function indicating that the thermodynamics is dominated 
by a single state. For continuous IRSB a second delta-function peak grows 
continuously away from the first as the transition temperature is crossed, the 
higher-g peak having a finite weight (1 — m), the lower a weight to. For the 
onset of discontinuous IRSB again there appears a second delta function peak 
but at a discontinuously larger value of q. This higher-g peak starts with a 
small weight (1 — m) which grows continuously as the temperature is lowered, 
with relatively much less change in the value of q. 

These observations suggest rather different images for the evolutions of 
free energy landscapes in systems with continuous and discontinuous RSB, 
both different from that of an RS system. The transition from paramagnet 
to RS order is envisaged as one in which the free energy evolves from a state 
with a single paramagnetic valley to one with a finite number of ordered states 
which are equivalent via global symmetry transformations (eg spin inversion) . 
For a continuous transition these ordered states grow smoothly from the para- 
magnetic state. For continuous RSB the transition is still continuous but to 
a thermodynamically relevant number ^ of non-equivalent ordered states. For 

'i.e. a number exponential in N. 
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IRSB these states are equally mutually orthogonal and their self-overlap grows 
continuously from their mutual overlap as the temperature is lowered; the mu- 
tual overlap is zero in the absence of a field. For full RSB (as in the SK model) 
these ordered states have a continuous range of mutual overlaps. For discontin- 
uous IRSB the ordered states appear with thermodynamic relevance already 
with large self-overlaps, and again with equal mutual overlaps, but their num- 
ber grows continuously as the temperature is lowered. One is therefore led to 
the conception that these states already exist above the transition temperature, 
with corresponding valleys in the energy landscape, but that their free energy 
relevance only becomes dominant when the transition temperature is reached 
and increases as the temperature is lowered. This already gives a hint also 
that their dynamical behaviour above the glass temperature might be different 
from systems with continuous RSB. 

Qiie example of a system which exhibits IRSB is the p-spin spherical spin 



glassES, whose Hamiltonian is given in (88). For fields less than a critical value 



he the onset of IRSB is discontinuous, for h > he it is continuous. Other 
examples axe found in Potts and quadrapolar glasses for high enough field 
dimension EH. Similar but self-ipduced behaviour is believed to control the 
glass transition in fragile glasses l3. 



3.10 Further subtleties 

Before leaving this replica analysis some further subtleties are worthy of note 
since they illustrate the need for caution in passing from conventional proce- 
dures to unconventional phase transition problems. 

The extremum of [F] giving the phase correctly is that maximizing [F] 
with respect to q, in stark contrast to conventional minimization. This is a 
consequence of the fact that the number of (a/3)-combinations, n{n — l)/2, 
becomes negative as n tends to zero. 

A second unconventional-looking feature is that if one analytically contin- 
ues the high temperature free energy [F], with m — q = 0, to low temperature, 
the result is lower than the free energy of the spin glass solution. In fact, 
however, the analytic continuation of a state beyond the limit of its validity is 
strictly meaningless. 

Within RSB theory there are several parameters against which to extrem- 
ize and the most obvious choice is not always correct; for example in IRSB 
one is tempted to extremizc with respect to qo, qi and the break point to. Al- 
though this works for continuous IRSB, for discontinuous IRSB it gives results 
different from those obtained from dynamics. 

In fact the relevant criteria are that systems must be stable against all 
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fluctuations and that configurational entropies must also be taken into account. 



4 Dynamics 

Thus far our discussion has concerned thermodynamics, analyzed from the 
perspective of the partition function as generator, albeit probed to yield insight 
more deeply into the metastable/pure state structure. Such analysis can be 
taken (and is still being taken) further to probe the state structure more deeply, 
for example by imposing overlaps between real replicas or by employing non- 
Boltzmann weights. 

However, often one would like to study dynamics. This is clearly neces- 
sary to learn about properties away from equilibrium, but it is not clear that 
spin glass systems do equilibrate. In the Introduction some experimental and 
simulational observations of preparation dependence and slow response were 
mentioned. In fact there are many more, of which one of the most intrigu- 
ing is the phenomenon of aging, where the response of a system depends on 
how long it has been waiting since preparation. Hence, it is interesting to 
discuss dynamics, allowing for the system to be away from equilibrium. ^ 
Recent studies have again exposed more interesting observations, implications, 
applications and conceptual challenges than could have been anticipated. 

Non-equilibrium dynamics is also a subject of considerable current interest 
in other areas of statistical physics, where systems without detailed balance 
in their macroscopic dynamics, and therefore no Boltzmann-like equilibrium 
even if a steady state is reached, show dynamically induced phase transitions. 
Here, however, I shall restrict discussion to spin-glass like systems. 

4-1 Microscopic dynamics 

The systems we consider are describable by Markovian microscopic dynamics; 
i.e. local in time and stochastic. Examples are 

(i) Ising spins satisfying either random sequential or parallel Glauber dy- 
namics in which spins are updated with probabilities 



""To aid this distinction the statistical mechanics of equilibrium is now often referred to the 
spin glass literature as 'statics'. 




where hi is the effective field experienced by spin i {hi — J^j Jij'^j foi' 
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(ii) Metropolis dynamics in which spins are chosen randomly and flipped if 
either the flip would lower the energy or with probability exp(— AiJ/T) 
if the energy change /S.E is positive.[] 



(iii) Soft spins obeying Langevin dynamics 

d(k _ ~dH 
dt d(j)i 

where rjiit) is white noise of variance T. 



mit) (83) 



We might note in passing that for a system to yield a thermodynamic 
equilibrium distribution of the Boltzmann type it iajiecessary for the dynamics 
to obey appropriate stochastic balance conditionsoEj. These are satisfied by 
(i) - (iii) with random sequential dynamics, symmetric interactions (J^ = 
— Jji) and no self-interaction {Ju — 0). For parallel-dynamics a different but 
also straightforward equilibrium distribution resultsEj. 



4-2 Macroscopic dynamics 

There are several ways to study the macroscopic dynamics. One is to try to use 
the microscopic equations to study directly the evolution of specified macro- 
scopic parameters. In general the equation of motion of one such parameter 
leads to a dependence on a higher one, whose equation of motion leads to 
still more in a never-terminating sequence. Well chosen order parameters and 
Ansiitze can yield closure but usually only approximately. In a later section 
we shall introduce one such procedure which appears to work quite well in 
comparisons with simulation. 

A second is to employ a generating functional in analogy with the partition 
function for thermodynamics. In this section I shall outline this procedure 
applied to a system with infinite-ranged interactions. 

The underlying principle is to introduce and study a generating functional 

^{A(i)}[i'^(^)}] = Tr^gf^^.^^S(Micms. eqn of motion) exp( J dt\{t)S{t)) (84) 

where (^(Micros, eqn. of motion) is a shorthand for the restriction of the 
microscopic variables to ones obeying the microscopic equations of motion. If 
necessary a Jacobian is also included to ensure Z^^^^-^^^y[{S{t)}] = 1. Then 

"Now we choose units with fc = 1. 
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any desired correlation or response functions may be obtained as derivatives, 
for example 



= lim 
A 



d d_ 

odKit) dx,{t') '""^i^} 



d d 



z 



{A} 



(85) 
(86) 

(87) 



The last identity follows from the restriction — 1- This ensures also 

that any disorder averaging can be performed directly on Z (without need 
for replicas). Sums of microscopic variables are then eliminated in favour of 
macroscopic variables by a dynamic analogue of the procedure of section (3.4) 
to yield a description in terms of an integral over macroscopic variables alone. 
For infinite-ranged models the integral is extremally dominated and extrem- 
ization yields coupled equations for macroscopic order parameters, correlation 
and response functions. 

Let us consider the process slightly more explicitly for a system of soft 
spins abeying Langevin dynamics, for example the p-spin spherical spin glass 
modeled in which 



H 



4l<l2---<'p 



the J are chosen independently randomly from a Gaussian distribution of vari- 
ance ,Pp\/2N'P~^ and the 4> satisfy the spherical constraint '^f = ^- The 
constraint can be implemented conveniently by adding a chemical potential 
term /i(i) ^ (/)^ to and determining /i(i) self-consistently. For simplicity we 
shall discuss explicitly only /i = 0. 

Before averaging over the stochastic noise 

/B BH 

Exponentiating the 5 functions introduces a new set of microvariables 



Z = j 50<5(^exp|^ 



dt 



dH 



-mit)) 



(90) 



Averaging over the stochastic noise, which is assumed to be Gaussian dis- 
tributed and local in i,t with variance T, yields 



{Z) = 



exp 



E 



mi 



dt 



dH 



) + T(t>dt)Mt) 



(91) 
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The quenched disorder enters through terms of the form 

«^P^E E Jn...^J^^it)■■■hi^)\ (92) 



t ii...ip 

where one of the is and the others are (j). Averaging over the disorder 
yields 

[{Z)]= J s^6$eMT.^Mt){^^ + f^Mt)} +TMt)Mt)] 

it 

+ z^'-^' E E (E {t)K it')) • • • (E k ii)k (*'))]} (93) 

t.t' comb ii ip 

where X^comb Enfi&ns a sum over all combinations in which one 4>{t) and one 
(j){t') are (l>{t), (pit') respectively and all the other are 4>. 

It will be noted that all the microscopic variables only enter in the macro- 
scopic form 4>i{t)(l)i{t') and hence they can be eliminated by use of 



5C{t, t')d{C{t, t') - J2 ^^(t)Mt')) 

i 

SG{t, t')6{G{t, t') - iV-i Yl ^^it)Mt')) 

i 

1 = / 5D{t, t')6{D{t, t') - N-^ kt)Mt'))- (94) 

i 

Further exponentiation of these (5-functions via auxiliary macrofunctions 
C{t, t'), G{t, t'),D{t, t') yields separable local Gaussian integrals in 0, 0, which 
are readily performed to yield 

[{Z)]= J SCSCSGSGSD6Dexp{-N<^{C,C,G,G,D,D)y, (95) 

Since the present purpose is only to illustrate procedures, and to save 
space, the explicit form of is not given here. A similar calculation can be 
applied to other soft-spin models, resulting in different but the following 
remarks continue to hold. Provided that N ^ t,t' the integral is dominated 
by the extremum which yields a set of coupled equations for the macroscopic 
order functions (of which only C(t,t') and G{t,t');t' < t are non-zero) ^ 

°The vanishing of D and G(t, t'); t' > t corresponds to causality. 
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These extremal order functions can be identified with the correlation and 
response functions 



i 

i i 

C{t,t') and G{t,t') satisfy coupled integro-differential equations, non-local 
in time. For the p-spin spherical model these are 



^Git,t')^ 5{t-t')~ti{t)G{t,t') 
at 



+ -p(p-l)/3V2 / dsG{t,s)C'P-'^{t,s)G{s,t') 
2 Jt' 



^ C{t, t') = - ti{t)C{t, t') + 2G{t',t) 



dt 



+ -Pip~l)l3^j^ I dsG{t,s)CP'^{t,s)G{s,t') 







1 

P{P~ 1) 

1 /■*' 

+ 7;PP^J^ dsCP"\t,.3)G{t',s) (97) 

2 Jo 

Above a dynamical transition temperature C and G are time-translational 
invariant (TTI), dependent only on the relative time r. They obey the usual 
fluctuation-dissipation theorem Gfdt{t) = —9{T)drCFDT[T). For the p-spin 
spherical model, and other models with discontinuous IRSB in their stat- 
ics, thej!_also lead to mode-coupling equations similar to those proposed for 
glasses Ed, with consequential fast /3-relaxation to a plateau followed by slow 
Qf-relaxation; 

(3 -relax : C(t) ~ qea + CpT^^; C > qea 

a -relax : C(r) ~ qea - Co^t-^-C < qea- (98) 

As the temperature is lowered, the plateau region grows until C(t) no longer 
decays to zero but remains a long time at a finite value Qea, corresponding 
to the qi, which appears discontinuously at the static transition. This defines 
a dynamical transition temperature^. For models whose statics has continu- 

PThis temperature is higher than that obtained by naive replica theory; a more sophisticated 
replica analysis based on marginal stability rather than extremization against m, or taking 
correct account of configurational entropy, gives the correct transition temperature. 
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ous RSB C(t) decays to zero, but with critical slowing down, at the replica 
theory transition temperature and the plateau qea grows continuously as the 
temperature is lowered further. 

Beneath the transition temperature a new behaviour is found in that both 
TTI and FDT break down at long relative times and aging is observed. This 
shows up for example in C(t + tu,,^™), where the times are measured from 
the time of preparation of the system at the temperature of study, after a 
quench from a high temperature state. When the relative time r is much less 
than the waiting time TTI and FDT hold and C settles to a plateau value 
Qea, corresponding to the self-overlap peak in the static P{q)- However, when 
T becomes of the order of i^, C {t + tw , tw) starts to decay again, reducing 
asymptotically towards zero. The phenomenon of dependence on tw is known 
as 'aging', the slow decay to zero as 'weak ergodicity breaking'. Thus 
C {t + , tui) is often expressed in terms of two parts 



C{t + )^CsT{T)+CAG{T + t^,t^,) (99) 

where Cst{t) is TTI and satisfies FDT, whereas Gag is the aging contribution. 
CsxiT) starts from its maximum value and decays rapidly towards a plateau 
at qea- Cag is zero for small r and goes asymptotically towards (—qEA) 
on a time-scale given by ti^; for the p-spin spherical model the decay is as n 

In the low temperature region there is a modified FDT 



G(M') = ^^. (100) 

At least for the p-spin spherical model X appears to depend on the times only 
via C 



Thus the system behaves as though it has an effective temnerMure T/X. 

Several of these properties are reminiscent of real glassesEJ'tJ and so similar 
procedures are currently being investigated for such systems, with rapid freeze- 
out of a glassy configuration providing the effective disorder controlling the 
remaining degrees of freedom. 

For a mom extensive review of dynamics away from equilibrium the reader 
is referred toE3 

■J A more general form is as ^2x >^[^>')^{'tw) / h\{T + tw)] 
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5 Beyond conventional spin glasses 

As indicated in tlie Introduction, concepts and techniques developed for spin 
glasses are now finding application and extension much more widely in the sci- 
ence of complex cooperative behaviour in disordered (or quasi-disordered) and 
frustrated many-body systems. In this section two such areas of application 
are introduced, namely neural networks and hard optimization. 

5.1 Neural networks: the spin glass approach 

The relevance of conventional spin glasses to neural networks lies not in any 
physical similarity, but rather in conceptual analogy and in the transfer of 
mathematical techniques developed for the analysis of spin glasses to the quan- 
titative study of several aspects of neural networks. This section is concerned 
with the basis and application of this transfer. 

Just as for spin glasses, neural networks also involve the cooperation of 
many relatively simple units, the neurons, under the influence of conflicting 
interactions, and they possess many different global asymptotic behaviours in 
their dynamics. In this case the conflicts arise from a mixture of excitatory 
and inhibitory synapses, respectively increasing and decreasing the tendency 
of a post-synaptic neuron to fire if the pre-synaptic neuron fires. 

The recognition of a conceptual relationship between spin glasses and re- 
current neural netjffiprks, together with a mathematical mapping between ide- 
alizations of eachEZi, provided the first hint of what has turned out to be a 
fruitful transplantation. 

In fact, there are now several respects in which spin glass analysis has 
been of value in considering neural networks for storing and interpreting static 
data. One concerns the macroscopic asymptotic behaviour of a neural network 
of given architecture and synaptic efficacies. A second concerns the choice of 
efficacies in order to optimize various performance measures. A third concerns 
the dynamics of associative recall, learning and generalization. 

We shall discuss networks suggested as idealizations of neurobiological 
stuctures and also those devised for applied decision making. We shall not, 
however, dwell on the extent to which these idealizations are faithful, or oth- 
erwise, to nature. 

5.2 Types oj Neural Network 

There are two principal types of neural network architecture which have been 
the subject of active study. 



33 



The first is that of layered feedforward networks in which many input neu- 
rons drive various numbers of hidden units eventually to one or few output 
neurons, with signals progressing only forward from layer to layer, never back- 
wards or sideways within a layer. This is the preferred architecture of many 
artificial neural networks for application as expert systems, with the interest 
lying in training and operating the networks for the deduction of appropriate 
few-state conclusions from the simultaneous input of many, possibly corrupted, 
pieces of input data. 

The second type is of recurrent networks where there is no simple feedforward- 
only or even layered operation, but rather the neurons drive one another col- 
lectively and repetitively without particular directionality. In these networks 
the interest is in the global behaviour of all the neurons and the associative re- 
trieval of memorized states from initializations in noisy representations thereof. 
These networks are often referred to as attractor neural networks Q They are 
idealizations of parts of the brain, such as cerebral cortex. Historically, they 
were the stimulation for devising artificial networks. 

Both of the above can be considered as made up from simple 'units' in 
which a single neuron received input from several other neurons which col- 
lectively determine its output. That output may then, depending upon the 
architecture considered, provide part of the inputs to other neurons in other 
units. 

Many specific forms of idealized neuron are possible, but here we shall 
concentrate on those in which the neuron state (activity) can be characterized 
by a single real scalar. Similarly, many types of rule can be envisaged, relating 
the output state of a neuron to those of the neurons which input directly to it. 
We shall concentrate, however, on those in which the efferent (post-synaptic) 
behaviour is determined from the states of the afferent (pre-synaptic) neurons 
via an 'effective field' 



where (f>j measures the firing state of neuron j, is the synaptic weight from 
j to i, and Wi is a threshold. For example, a deterministic perceptron obeys 
the output-input relation 



where <j)i is the output state of the neuron and f{h) is a non-linear sigmoidal 
function. 

''They are often abbreviated as ANN, but we shall avoidc this notation since it is also 
common for artificial neural networks 




(102) 



0- = nh,), 



(103) 
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It is often convenient technically to specialize ftirther to binary-state 

(McCuUoch-Pitts) neurons, taken to have (pi ai = ±1 denoting firing/non- 
firing. One cannot then have a deterministic sigmoidal response but instead 
one can interpret the sigmoidal ftmction, suitably normalized, as a measure of 
the probability of firing. In the spirit of picking out the essentials it is convenient 
to employ a sigmoid function characterized by a single rounding parameter. 
One such choice is the Glauber rule. 

(Tj a'i with probability ^[1 + tanh(/3/ijCrj')], (104) 

where now T = f3~^ is a measure of the degree of stochasticity or rounding 
of the initial sigmoidal response function, with T = (/3 = oo) corresponding 
to determinism. In a network of such units, updates can be effectuated either 
synchronously (in parallel) or randomly asynchronously; we shall concentrate 
on the latter which is more analagous to a spin glass. 

Thus we have a system reminiscent of the Ising spin glass model discussed 
earlier. The similarities are that neuron states are analagous to the microscopic 
spin states and the synaptic efficacies are analagous to the exchange interac- 
tions . We expect that at high T both will be effectively paramagnetic, with 
unconfined dynamics. Our experience with spin glasses tells us that frustrated 
interactions lead to a low temperature free energy surface with many valleys, 
which immediately suggests that a similar frustration in synaptic efficacies 
will be needed to produce many global attractors for the neural dynamics, to 
act as memory storages. Hence the need for both excitatory and inhibitory 
synapses; Jij both positive and negative. We shall see that the memories are 
related to global microstates. The desire that the dynamics is driven to those 
microstates implies that for neural networks the Jij must be trained or tuned 
to put the 'valleys' in the correct places; this is the first difference from spin 
glasses. A second is that these valleys should have reasonably large basins of 
attraction, subject to the constraints imposed by having many attractors. A 
third is that our interest in neural networks is strictly in the attractor space 
for the dynamics rather than the thermodynamics of a Boltzmann-weighted 
Hamiltonian. 

Thus we arrive at a mental image of a recurrent neural network as a sys- 
tem of many neurons in which patterns are characterized by particular global 
microstates cr = ^^;cr = (Ji, (T2--0"Ar; /i = where the n label the pat- 

terns, and which is capable of retrieving those patterns associatively from a 
noisy start ?7^; rj^ = Ci£,^ where q = ±1 with a probability (1 — p,p) where p 
characterizes the initial noise in the input information. 

A potential scenario is to choose a system in which the Jij are such that, 
beneath an appropriate temperature (stochasticity) T, there are p disconnected 
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basins, each having a macroscopic overlap |j with just one of the patterns and 
such that if the system is started in a microstate which is a noisy version of a 
pattern it will iterate towards a distribution with a macroscopic overlap with 
that pattern and perhaps, for T 0, to the pattern itself]^ To store many non- 
equivalent patterns clearly requires many non-equivalent basins and therefore 
competition among the synaptic weights 

The mathematical machinery devised to study ordering in random magnets 
is thus a natural choice to consider for adaptation for the analysis of retrieval 
in the corresponding neural networks. An introduction to this adaptation is 
the subject of the next section. However, before passing to that analysis a 
further analogy and stimulus for mathematical transfer will be mentioned. 

This second area for transfer concerns the choice of {Jij} to achieve a 
desired network performance. Provided that performance can be quantified, 
the problem of choosing the optimal {Jij} is equivalent to one of minimizing 
some effective energy function in the space of all {Jij}- The performance 
requirements, such as which patterns are to be stored and with what quality, 
impose 'costs' on the Jij combinations, much as the exchange interactions 
do on a specific choice of the spins in (||), and there are normally conflicts 
in matching local with global optimization. Thus, the global optimization 
problem is conceptually isomorphic with that of finding the ground state of a 
spin glass and a conceptual and mathematical transfer has proved valuable. 

5.3 Statistical Physics of Retrieval 

In this section we consider the use of spin glass techniques for the analysis of 
the retrieval properties of simple recurrent neural networks. 

Let us consider such a network of N binary-state neurons, characterized 
by state variables cr^ — ±l,i = 1...7V, interacting via stochastic synaptic oper- 
ations and storing, or attempting to store p patterns {ff } = {±1}; A* = 1, ■■■P- 
Interest is in the global state of the network. Its closeness to a pattern can be 
measured in terms of the corresponding (normalized) overlap 



"A precise definition of overlap is given later in ( |l05[ ). With the normalization used there 
an overlap is macroscopic if it is of order 1 . 

*For a network to iterate at T = precisely to a desired pattern requires an optimized design 
of the {Jij}. This is not necessarily the best design for retrieval at T ^ 0^^ 
"Note that this concept applies even if there is no Lyapunov or energy function. The 
expression 'basin' refers to a restricted microscopic phase space of the {cr}, even in a purely 
dynamical context. 




(105) 
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To act as a retrieving memory the phase space of the system must separate 
so as to include effectively non-communicating sub-spaces each with macro- 
scopic 0{1) overlap with a single pattern. 

5.4 The Hopfield model 

A particularly interesting example for analysis was proposed by HopfieldEl. It 
employs symmetric synapses Jij = Jji, no self-interaction {Ju = 0) and ran- 
domly asynchronously updating dynamics, leading to the asymptotic activity 
distribution (over all microstates) 

pia) ^ exp((-/3i?(rT)); /3 = T'^ (106) 

where E{cr) has the form of (|2|). This permits the applications of the ma- 
chinery of equilibrium statistical mechanics " to study retrieval behaviour and 
develop conceptual understanding and quantification. In particular, one stud- 
ies the resultant thermodynamic phase structure with particular concern for 
the behaviour of the to''. 

Hopfield further proposed the simple synaptic form ^ 

^"'ECej'(l-'5^,), (107) 

inspired by the observations of Hebb. Let us turn to the analysis and impli- 
cations of this choice, with all the thresholds {Wi} taken to be zero and for 
random uncorrelated patterns 

For a system storing just a single pattern, the problem transforms imme- 
diately, under ai — > (Ji£,i, to a pure ferromagnetic Ising model with = N^^. 
The solution is well known and m satisfies the self-consistency equation 

m = tanh(/3TO), (108) 

with the physical solution to = for T > 1 (/3 < 1) and a symmetry-breaking 
phase transition to two separated solutions ±|to|, with to 7^ 0, for T < 1. 

For general p one may express exp(— /3i?(cr)) for the Hopfield-Hebb model 
in a separable form. 

p TV 

exp{-(]E{cT)) = exp[+(/3/2iV) 5](^ ^^a.)' - (3p/2] 

fi—l i—l 

"This is the reason for the interest in employing symmetric synapses even though this is not 
a requirement for neural storage nor universal biological reality. 

'^Note that this is infinite-ranged; hence (possibly sophisticated) mean field theory will be 
applicable. 
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f nPd77~i^(^)^exp[f](-7V/3(m'^)V2-/3m''^a,C)-/3p/2], (109) 



where we have employed (|2 

Because the only term in (Ji in (109) has the separable form 11^ exp(— OifJi) 
the sum on {(Ti} in Z is now straightforward and yields 

/ (n^fim^(/37V/27r)5)exp(-Ar/3/({m'^})) (110) 



where /{{m"}) = ^(m^)V2 - (A^/9)~^ ^ ln[2 cosh/3 ^ m'^Cl'] + p/2N. 

/i=l i IJ,=1 

(111) 

Intensive numbers of patterns: 

If p is finite (does not scale with N), /({m^}) is intensive in the ther- 
modynamic limit {N — > oo) and the integral is extremally dominated. The 
minima of / give the self-consistent solutions which correspond to the stable 
dynamics, although only the lowest minima are relevant to a thermodynamic 
calculation. The self-consistency equations are 

TO^ = 7V-i^4''tanh(/3^TO'^^n = m'' (112) 
yielding the results: ^ 

1) T = 0: (i) All embedded patterns are solutions; ie. there are p solutions 
(all of equal probability) 

m'^ = 1; one = : rest 

(ii) There are also mixed solutions in which more than one is non-zero. 
Solutions of type (i) correspond to memory retrieval, while hybrids of type (ii) 
are normally not wanted. 

2) 0.46 > T > 0: There are solutions 

(i) m'' = m 7^ 0; one (x =0; rest 

(ii) mixed states. 

In the case of type (i) solutions, we may again speak of retrieval, but now it is 
imperfect. 

3) 1 > r > 0.46: Only type (i) solutions remain, each equally stable and with 
extensive barriers. 

4) T > 1: Only the paramagnetic solution (all — 0) remains. 

^The results are quoted for the limit — > oo. For finite but large A'^ read 
VO" = "0(1)", "0" = "<0(1)". 
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Thus we see that retrieval noise can serve a useful purpose in ehminating 
or reducing spurious hybrid solutions in favour of unique retrieval. 



5.5 Extensive numbers of patterns: 

The analysis of the last section shows no dependence of the critical temper- 
atures on p. This is correct for p independent of N and N oo. However, 
even simple signal-to-noise arguments demonstrate that interference between 
patterns will destroy retrieval, even at T = 0, for p large enough and scaling 
appropriately with N. Geometrical, information-theoretical and statistical- 
mechanical arguments in fact show that the maximum pattern storage allow- 
ing retrieval scales as p — aN, where a is an iV-independent storage capacity. 
Thus we need to be able to analyse retrieval for p of order N, which requires 



a different method than that used in (11C),(111). One is available from the 
replica theory of spin glasses. 

As noted earlier, physical quantities of interest are obtained from In Z. 
This will depend on the specific set of {Jij}, which will itself depend on the 
patterns {S,^} to be stored. Statistically, however, one is interested not in 
a particular set of {Jij} or {^f} but in relevant averages over generic sets, 
for example over all sets of p patterns drawn randomly from the 2^ possible 
pattern choices. Furthermore, the pattern averages of most interest are self- 
averaging, strongly peaked around their most probable values. Thus, we may 
consider {\nZ)^^y where ( means an average over the specific pattern 
choices. 

The methodology is analgous to that employed in our replica study of spin 
glasses. In place of the ferromagnetic order parameter m one now has all the 
overlap parameters m'^. However, since we are principally interested in retriev- 
ing symmetry-breaking solutions, we can concentrate on extrema with only 
one, or a few, macroscopic {0{N^)) and the rest microscopic (< 0{N~2)). 
This enables us to obtain self-consistent equations for the overlaps with the 
nominated (potentially macroscopically overlapped or condensed) patterns 

= J2 - 1, • • • s, (113) 

i 

where the 1, ...s label the nominated patterns, ( )t denotes the thermal aver- 
age at fixed {^}, and ( denotes an average over the other (uncondensed) 
patterns, together with a spin-glass like order parameter 

g=(iV-i^(a,)|){e} (114) 
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and a mean-square average of the overlaps with the un-nominated patterns. 
Retrieval corresponds to a solution with just one m'^ non-zero. 
Explicitly, averaging over random patterns yields 

(Z") = cxp(-np/3/2) J2 I n^n^{d™^"(/3iV/27r)iexp[-Ar(/3^(TO''")V2 

-h7V~i^lncosh(/?^m'^"cr,n)]}. (115) 

To proceed further we separate out the condensed and non-condensed pat- 
terns. For the non-condensed patterns, fi > s, only small to'* contribute and 
the corresponding In cosh can be expanded to second order to approximate. 

n^>,exp[-^lncosh(/?^m'^"af )] ^ n^>, exp[^ ^ m'^^m^'' ^ af ]. 

i a a.(3 i 

(116) 

X] erf erf may be effectively decoupled by the introduction of a spin-glass like 
order parameter q"^^ via the identities 

1 = j dq'^'^Siq"'^ - N-^ ^'^f ) 

i 

= I dq-^ J g"' exp(^r^(,"^ - ))' (117) 



whence ( |116D becomes 



J n(«,)d9"''|" exp(z Y - Y ) 



M>s (a/3) 



(a/3) 

' 2 



N(3^ Y Y (f^rn'^'^'n^'"' + ^ I] (US) 



In ( 115 ) the rn^";/i > s integrations now yield the cr-independent result 

(27r/iV/3)^(p-^(det A)-^(p-'^), where 

A'^P = (1 - /3)Ja/3 - (119) 
while the Gi contributions enter in the separable form 

s 

e^P[E(E lncosh(/3^ m^Vf) - iN'^ ^ r'^^^rf )]. (120) 

i 11=1 a (a/3) 
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Further anticipating the result that the relevant q scales as p — aN has the 
consequence that re-scaling q — i0^pr 

- (/3iV/2^)"/2 j n;:,„^idm'- j n(„^)dg"'^dr"'5e-^'5* (121) 

where $ is intensive given by 

$({m"},{g"''},K'^}) 

= + i y y (^A-)2 + (Pjlil Tr In A + a/3 y r"'^^"^ 
27V 2^-^^ ' 2(3N f-^, 

a ^=1 (a:/3) 

s 

-/?-Mn^ exp{^lncosh(/3^m^"CT") + a/3^r"'3cr"tT^}. (122) 

M=l (a/3) 



(121) is thus extremally dominated. At the extremum 

m'^" = A^-'E(^fO* (123) 

■i 

g"^ = iV-i^«af)* ;a^/3 (124) 

■i 

;a^/3. (125) 

Within a replica-symmetric ansatz m^" = m^, g"'' — g, r"^ — r, self-consistency 
equations follow relatively straightforwardly. For the retrieval situation in 
which only one is macroscopic (and denoted by m below) they are 

dz 

exp(-zV2)tanh[/3(zVc^ + TO)] (126) 



27r 

/dz 
^=exp(-zV2)tanh2[/3(zy^ + m)] (127) 
V27r 

r = g(l-/3(l-g))-2 (128) 

Retrieval corresponds to a solution m ^ 0. There are two types of non- 
retrieval solution, (i) m = 0, g = 0, called paramagnetic, in which the system 
samples all of phase space, (ii) to = 0, g 7^ 0, the spin glass solution, in which 
the accessible phase space is pstricted but not correlated with a pattern. Fig. 
10 shows the phase diagramed; retrieval is only possible provided the combi- 
nation of stochastic noise T and pattern interference noise a is not too great. 
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Figure 10: Phase diagram of the Hopfield model (after Amit et. al ^^). Tm indicates the hinit 
of retrieval solutions, between Tm and Tg there are spin-glass like non-retrieval solutions, 
above Tg only paramagnetic non-retrieval. Tc denotes a thermodynamic transition, at which 
retrieval states become lowest in free energy, rather than just attractors. Tn shows the limit 
of replica-symmetry breaking in the retrieval phase. 

There are also spurious solutions with more than one m'' ^ 0, but these are 
not displayed in the figure. 

As in the case of the SK spin glass one can check for instability against 
replica symmetry breaking. RS turns out to be unstable in the spin glass region 
and in a small part of the retrieval region of (T, a) space near the maximum 
a for retrieval. However RSB gives rise to only relatively small changes in the 
critical retrieval capacity. 

5. 6 Statistical Mechanics of Learning 

In the last section we considered the problem of assessing the retrieval capabil- 
ity of a system of given architecture, local update rule and algorithm for { Jy }. 
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Another important issue is the converse; how to choose/train the {Jij}, in or- 
der to achieve the best performance. Various such performance measures are 
possible; for example, in a recurrent network one might ask for the best over- 
lap improvement in one sweep, or the best asymptotic retrieval, or the largest 
size of attractor basin, or the largest storage capacity, or the best resistance 
to damage; in a feedforward network trying to learn a rule from examples one 
might ask for the best performance on the examples presented, or the best 
ability to generalize. Statistical mechanics as developed for spin glasses has 
played an important role in assessing what is achievable in such optimization 
and also provides a possible mechanism for achieving such optima (although 
there may be other algorithms which are quicker to attain the goals which have 
been shown to be accessible). 

Thus in this section we discuss the statistical physics of optimization, as 
applied to neural networks. Similar techniques apply to other optimization 
problems. 

Statistical physics of optimization 

Consider a problem specifiable as the minimization of a function E^a}{{b}) 
where the {a} are quenched parameters and the {6} are the variables to be 
adjusted, and furthermore, the number of possible values of {6} is very large. 
In general such a problem is hard. One cannot try all combinations of {6} 
since there are too many. Nor can one generally find a successful iterative 
improvement scheme in which one chooses an initial value of {b} and gradually 
adjusts the value so as to accept only moves reducing E. Rather, if the set {a} 
imposes conflicts, the system is likely to have a 'landscape' structure for E as 
a function of {b} which has many valleys ringed by ridges, so that a downhill 
start from most starting points is likely to lead one to a secondary higher- ii^ 
local minimum and not a true global minimum, or even a close approximation 
to it. 

To deal with such xiroblems computationally the technique of simulated 
annealing was invented cll. It simulated the thermal excitation procedure used 
by a metallurgist to anneal out the defects which typically result from rapid 
quenches. Specifically, one treats £^ as a microscopic 'energy', invents a com- 
plementary annealing temperature Ta, and simulates a stochastic thermal dy- 
namics in {6} which, in principle, iterates to a distribution of the Gibbs form 

p{{b})^exp{-E{{b})/TA), (129) 

Then one reduces Ta gradually to zero, to try to achieve the ground state. 

The actual dynamics has some freedom - for example, for discrete vari- 
ables Monte Carlo simulations with either a heat bath or a Metropolis algo- 



rithm both lead to (129). For continuous variables Langevin dynamics with 
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^ = —\/fjE{b) + ri{t), where ri{t) is white noise of strength Ta, would also 
be appropriate. Of course, for a frustrated and disordered E{{b}) the slow 
relaxation and non-equilibration features discussed earlier make it difficult to 
achieve the true minimum in practice. 

Computational simulated annealing is used to determine specific {b} given 
specific {a}. It is also of interest, however, to consider generically what is 
achievable, averaged over all equivalently chosen {a}. 

Hence we turn to the analytic equivalent of simulated annealing. We define 
a generalized partition function 

Za = J2 eM~E{{b})/TA), (130) 
{b} 

from which the average 'energy' at temperature Ta follows from 

{E)T.^~-^\nZA; Pa^T^' (131) 

and the minimum E from the zero 'temperature' limit, 

E^i^ = lim {E)t^. (132) 

As noted earlier, we are often interested in typical behaviour, as charac- 
terized by averaging the result over a random choice of {a} from some charac- 
terizing distribution. Hence we want {lnZA){a}, which naturally suggests the 
use of replicas again. 

In fact, the replica procedure has been used to study several Ip^jd-cmnbi- 
natorial optimization problems, such as various graph partitioningcaeSa and 
travelling salesman problems. Here, however, we shall concentrate on neural 
network applications. 

Cost functions dependent on stability fields 

One important class of training problems for pattern-recognition neural 
networks is that in which the objective can be defined as minimization of a 
cost function dependent on patterns and synapses only through the stability 
fields; that is, in which the 'energy' to be minimized can be expressed in the 
form 

Et,}iU}) = -EE5(Ar); Ar =erE^^^-^"/(E4)*- (133) 

Before discussing general procedure, some examples of g{A) might be in 
order. 
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The original application of this technique to neural networks concerned 
the maximum capacilji for stable storage of patterns in a network satisfying 
the local update rulell3 

a[= sgn(^Jya,). (134) 

Stability is determined by the Af ; if Af > 0, the input of the correct bits of 
pattern ^ to site i yields the correct bit as output. Thus a pattern /i is stable 
under the network dynamics if A^ > 0; all i. A possible performance measure 



is therefore given by (133) 



5(A) = -e(-A). (135) 

(7(A^) is thus non-zero (and negative) when pattern /i is not stably stored at 
site i. Choosing the {Jy } such that the minimum E is zero ensures stability. 
The maximum capacity for stable storage is the limiting value oi p/N ~ a for 
which stable storage is possible. _ 

An extension is to maximal stabilityl£3. In this case the performance mea- 
sure employed is g(A) = — 0(k — A) and the search is for the maximum value 
of K for which Emin can be held to zero for any capacity a, or, equivalently, 
the maximum capacity for which Emin = for any k. 

Yet another example is to consider a system trained to give the greatest 
increase in overlap with a pattern in one step of the dynamics, when started 
in a state with overlap rrit For the update rule ai ^ a'^ — sgn {hi + Tz) where 
z is stochastically Gaussian-random, the appropriate pe 
averaged over all specific starting states of overlap rrit, isc 



brmance function, 



5(A).erf l^^d^l. (136) 

Methodology 

Let us now turn explicitly to the analytic minimization of a cost function 
of the general form 

N 

with respect to Jj which satisfy spherical constraints ^ Jj = N. The {^} are 
random quenched ±1. The result is expressed averaged over the choice of the 
{f}. Thus we require {\nZ^^y)^^y where 

= J n,dJ5(E Jf - N) expiPA E ai^ni (138) 
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In order to evaluate the {^} average we separate out the explicit ^ dependence 

in g{A) via delta functions 5(A'^ — j Jj^j /^^) ^^'^ express all the delta 
functions in exponential integral representation, 

%}= J UjdJj J ^exp(ie(5]j|-7V))n^y' dX^^eMM^")) 

exp(z0^(A'' -eYl -^j^j/^'^yy (139) 

j 

Replica theory requires and therefore the introduction of a dummy 

replica index on each of the J, e, A and ((>; wc use a = 1, ...n. For the case in 
which all the ^ are independently distributed and equally likely to be ±1, the 
(, average involves 

{expi-ir"e E -^."^i = cos(E J^TV-^). (140) 

j a 

For large A'' (and 0J < 0(1)) the cosine can be approximated (after expansion 
and re-exponentiation) by 

cos{^cl)^"'J^N-i) = exp(-(E</>''«Jf)V2iV). (141) 

a a 

Thus 

{z-)ii}= J n„^n,dJ«n^dA''"g'"'exp$({n,{j;},{A'^"},{<^^"}) 

(142) 

where 



y I III- ^ ^ 



-EE E -^Pj^ + PAY.9{>^n- (143) 

H a</3 j fta 

We eliminate the term in J2j '^f'^j favour of a spin glass like order parameter 

1 = / dq"^S{q"^ - N-'^ E Jpj) (144) 

j 

= {N/2-k) I dx'^'^dq"'^ exp{ix"^{Nq"f^ - E Jpj))- (145) 
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The j = I, ...N and fi — 1, ...p contribute only multiplicatively and 

n.<;3^rfg"^n„— exp[7V( - 9"''^"'' + ^./({e"}, {^'"'}) 

a</3 

+ {p/N)G^{{q-f>}})]; 
expGj({e"},{a;"'3}) = / H^dJ" exp(- ^ e"(( J")^ - 1) + ^ x"^ J" J'^), 

expG^({<z"n)= / n„dA"g"exp(^[/3^g(A")+*A"r-(0")V2-5] 



a</3 

(146) 



a</3 

(147) 

Since (/j and are intensive, as is p/N — a for the situation of interest, (146) 
is dominated by the maximum of its integrand. 

In the rephca symmetric ansatz e" = e, x"f^ = x and q"^ — q. In the hmit 
n — > 0, ehmination of e and x at the saddle point d^/de = d^/dx ~ d^/dq ~ 
yields 

(lnZ){^} = N ext,(i ln[2^(l - q)] + (2(1 - q))-' 

+a J Dt\n{2n{l-q))-i J dXexp[PA9i\) - (X - Vqif /2il - q)]) (US) 
where Dt = dt exp{~t'^/2) / V2^. (149) 

In the low temperature limit. Pa — > oo, g — > 1 and /^^(l ~ q) ~^ 7i inde- 
pendent of Ta to leading order. The integration over A can then be simplified 
by steepest descent, so that 



/ 



dAexp(/3^5(A)-(A-i)V2(l-<z)) ^ exp(/3^(5(A(t))-(A(t)-i)V27)) (150) 



where X{t) is the value of A which maximizes (.g(A) — (A — t)'^/2^); i.e. the 
inverse function of t{X) = X — jg' iX)\)^^^- Extremizing with respect to q gives 
the (implicit) determining equation for 7 

DtCX{t) - t)^ = a (151) 



The average minimum cost follows from 



d 

{E,nin) i^} = lii-n — -(lnZ){^.}. (152) 

PA^oci OpA 
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Similarly, any measure /(^^))t4){c} may be obtained from < InZ > 

by means of generating the functional procedure. Alternatively, they follow 
from the local field distribution p(A) defined by 

p(A) = ((p-^E<5(A-A^))tJ{4) (153) 

which is given by 

p{X) = J DtS{X-X{t)). (154) 

Just as in the replica analysis of retrieval, the assumption of replica sym- 
metry for q"^ needs to be checked and a more subtle ansatz employed when 
it is unstable against small fluctuations q"^ ^ q + r]°'^]r]°'f^ small. In fact, it 
should also be tested even when small fluctuations are stable (since large ones 
may not be). Such effects, however, seem to be absent or small for many cases 
of continuous {Jij}, while for discrete {Jij} they are more important]^. 

Learning a rule 

So far, our discussion of optimal learning has concentrated on recurrent 
networks and on training perceptron units for association of given patterns. 
Another important area of practical employment of neural networks is as expert 
systems, trained to try to give correct decisions on the basis of many observed 
pieces of input data. More precisely, one tries to train a network to reproduce 
the results of some (usually-unknown) rule relating many-variable inputs to 
few-variable outputs, on the basis of training with a few examples of input- 
output sets arising from the operation of the rule (possibly with error in this 
training data). 

To assess the potential of an artifical network of some structure to re- 
produce the output of a rule on the basis of examples, one needs to consider 
the training of the network with examples of input-output sets generated by 
known rules, but without the student network receiving any further informa- 
tion, except perhaps the probability that the teacher rule makes an error (if it 
is allowed to do so). 

Thus let us consider first a deterministic teacher rule rj = V{{^}), relating 
A'' elements of input data {£,i , £,2 ■■■£,n) ^'^ ^ single output rj^, being learned 
by a deterministic student network rj = i?({^}). B is known whereas V is 
not. Training consists of modifying B on the basis of examples drawn from 
the operation of V. Problems of interest are to train B to give (i) the best 

ffor discrete {Jij} there is first order replica-symmetry breaking^^ at a temperature liigher 
than that at which there is instabihty against small fluctuations. 
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possible performance on the example set, (ii) the best possible performance on 
any random sample drawn from the operation of V, irrespective of whether it 
is a member of the training set or not. The first of these refers to the ability of 
the student to learn what he is taught, the second to his ability to generalise 
from that training. 

The performance on the training set /i = 1, ...p can be assessed by a train- 
ing error 

ii;,^f]e(i?(U''}),y({n)) (155) 

where e{x, y) is zero ii x — y, positive otherwise. 

A common choice for e is quadratic in the difference {x — y). With the 
scaling e{x,y) = {x — yY/^ one has for binary outputs, rj — ±1, e{x,y) = 
Q{—xy), so that if B{{£^}) is a perceptron, 

Bm)^ sgn(^J,^,), (156) 

j 

then e'' = 9(— A'^) where now A'^ = rj'^ J2j Jj^j /{J2j ^j)^ ' making Ct analgous 



to E"^ of ( |133| ) with 'performance function' (135). This we refer to as minimal 
stability learning. Similarly, one can extend the error definition to e'^ = Q{k — 
A'^) and, for learnable rules, look for the solution with the maximum n for zero 
training error. This is maximal stability learning. 

Minimizing et can proceed as discussed above, either simulationally or an- 
alytically. Note, however, that for the analytic study of average performance 
the (f?,C) combinations are now related by the rule V, rather than being com- 
pletely independent. The generalization error Cg — {e{B{{^}),V{{^}))s follows 
from the resultant distribution p{A). 

6 Dynamical replica theory 

Let us now turn to a different approach to macroscopic dynamics, namely an 
attempt to find good approximate (or exact) closed autonomous flow equations 
for macaxjsepoic observables, starting from instantaneous stochastic micrody- 
namics EaOEJ. 

Specifically, let us concentrate on a system with Ising variables obeying 
random sequential Glauber dynamics as in 4.1 (i). With appropriate coarse- 
graining this leads to the master equation 

, N 

-pticr) = J2 [ptiFkCT)WkiFkCT) ^ pti<T)Wki<T)] (157) 

fe=i 
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where Fk is the spin-flip operator 

Fk^er) = -at, <Jn): (158) 

Wk{cr) is the transition rate 

Wkicr) = i[l - (7fetanh(/3/ifc(cr))], (159) 

and we are again using the vector notation er = (cti, a^)- 

From ( |157|) we may derive an equation for the evolution of the macrovari- 
able probabihty distribution 

Pt[f2] ==^Pt(*T)<5[j^-fi(<T)]; n = {nu-n,,) (leo) 

(T 

in the form 
where 

(162) 

and the notation ()o;t refers to a sub-sheU average 

In several cases of interest and for finite times, only the first term on the 
right hand side of ( |161[ ) survives in the limit iV — > oo, yielding the deterministic 
flow 

jnt^{J2w.{a)[niF,a)-n{a)])^.^^ . (164) 

i 

In general this does not yet constitute a closed set of equations due to the 
appearance of Pt(cr) in the sub-shell average. However, we may attempt to find 
an appropriate choice of for which closure may be attained either exactly or 
approximately. Ideally we would like fl to be as low-dimensional as possible. 
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One particularly simple example occurs for the case of an infinite range 
Ising ferromagnet; Jij = Jq/N . In this case the magnetization m = N^^ (Ji 
suffices alone as a macrovariable whose evolution is deterministic and closed, 

—TO = tanh(/3 Jom) — m, (165) 

and yields the usual mean field solution in the steady state limit 

—TO = TO = tanh(/3( Jqto)). (166) 

A greater challenge is posed by problems with sufficient disorder and frus- 
tration, such as those given by 

J^J = Jo/N + Jz,j/VN; (z,,) - (z,/) = 1; ]. (167) 

where Zij is a quenched random parameter. This is the case for two particular 
model problems of interest; the Sherrington-Kirkpatrick (SK) spin glass and 
the Hopfield neural network. In the SK model the {z^} arc chosen randomly 
from a Gaussian distribution. In the Hopfield model, concentrating for simplic- 
ity on the region of phase space within the basin of attraction of one pattern, 
taken arbitrarily as = 1, it is convenient to apply the gauge transformation 



o'iS.i, Jij (.i^jJij, to re-write (107) in the form above with 
1 



= ^Z2 ^'^f ^l^.'' Jo^l J = V^ (6) = . (168) 



It is straightforward to show that to — N^^ is insufficient for a closed 

macroscopic evolution for finite J, although it does suffice if J ^ as ^ oo, 
as is the case for a Hopfield model with only one condensed pattern, storing 
only a less than extensive number of patterns (limTv^oo a = 0). But how many 
macrovariables does one need and what are they? 

Before giving an answer to the last question which appears to be at least 
very close to the truth, let us consider an intermediate step which is useful 
illustratively. Although our analysis applies to finite times, it is instructive to 
ask first about the long time steady state, assuming equilibration. For problems 
with detailed balance in their dynamics one knows that in the limit as t — > oo 
before N oo the microstate distribution takes the Boltzmann form Poo{cr) ^ 
exp(— /3i7). This is the case in the above examples which have Jij = Jji, 
yielding the Hamiltonian 

i7/A^ = -Af-i^ Jya,a, = -^Jom^fr) - Jr{(T) + 0{N-^) (169) 

i<j 
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where 

r(,T) = iV-3/2^a,z,,aj. (170) 

Thus, as long as r{cr) ^ 0{1) it cannot be ignored in the set of Jl. It is 
straightforward to show that r{cr) ^ 0{\) for both the SK spin glass and the 
Hopfield model at finite storage ratio a. Thus we shall first discuss an attempt 
to find a non-equilibrium macrodynamics in terms of m, r, alone, and show 
that it provides a reasonable but imperfect description. We shall then go on 
to a more sophisticated theory in terms of a generalized order function which 
provides a very good fit to the results of microscopic simulation. 



6. 1 The simple version of the theory: two order parameters 
In this section we choose the minimal form 



= {ni{(T),n2{(T)) = (m(a),r(a)) 



(171) 



The resultant i-t[r2*] does indeed satisfy a Liouvillc equation in the thermody- 
namic limit, yielding the deterministic flow equations 



dm 
~dt 

dr 

'dt ~ 



dzDm,r;t{z) tanh /3(Jom -f Jz) — m 
dzD„ij,-tiz)ztaTi]iP{Jom + Jz) — 2r 



(172) 



(173) 



where Dm,r;t{z) is the sub-shell averaged distribution of the disorder contribu- 
tions to the local fields 



Dm,r-t{.z) = lim 



Y.cTPt{<T)5{m - m{a))5{r - r{a))N-^ 5{z - z,{a)) 
Y.(TPt{(^)^{'>T^ - m(cr))(5(r - r{cT)) 

(174) 

2,(cr) = A^-i/^^Zy-a, . (175) 



h,{(T) = Jom{(T) + Jz^{(t) + 0{N-^) 



As yet, because of the pt{cr) in (174) , equations (172) and (173) are not 
closed except in the disorder- free case J = 0. To close the equations we intro- 
duce two simple Ansatze: (i) we assume that the evolution of the macrostate 
(to, r) is self-averaging with respect to the specific microscopic realization of the 
disorder {zij}, (ii) as far as evaluating D{z) is concerned we assume equiparti- 
tioning of the microstate probability pt(cr) within each {m,r) shell. The first 
of these Ansatze is well borne out by computer simulations of the microscopic 
dynamics and permits averaging D{z) over pattern choices. The second, which 
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is clearly true as t oo since Poo(c) depends only on m and r but can only be 
judged a posteriori for general time, eliminates memory effects beyond their re- 
flection in m, r and removes explicit time-dependence from D. Together these 
Ansatze give 

n n f ./'J:aSim~mia))S{r~rifT))N-^J:^6{z-^^H) 



J2(T S{m - m{(T))S{r - r{(7)) 

(176) 



where y indicates an average over the quenched randomness. This 



yields closure of ( |172| ) and (|173|) since Dm,riz) now depends only upon the 
instantaneous values of m, r and no longer on other microscopic measures of 
history. 



The actual evaluation of Dm.r{z) from (176) remains a non-trivial exercise, 
but one which is amenable to solution by replica theory Xas developed for 
the investigation of local field distributions in spin glasses Ej). After several 
manipulations it can be expressed in the form 



A„,.(2)=nmyn n dxtdyfeM~N'i>{m,r,z-{x'^},{yf})] (177) 

i^j a,/3— l...n 

where the number of indices i,j is finite and <& is 0{N^). Because the argu- 
ment of the exponential scales as N , the integral can be evaluated by steepest 
descents. 

The extremization is complicated and in its complete form involves sig- 
nificant subtleties, including an extension of those devised by Parisi-Eor the 
analysis of the spin glass problem. It is discussed in detail elsewhere E3Eil; here 
we note only a few salient results. Important among them is that in the steady 
state limit of dm/ dt = dr/dt = the analysis yields the full thermodynamic 
results obtained from equilibrium analysis, including replica-symmetry break- 
ing. For more general times explicit analysis to date has only been completed 
within the further Ansatz of replica-symmetry in the dynamic analogue of the 
spin glass order parameter g"'' which enters into the evaluation of Dm^riz), 
but including a determination of the limit of its applicability against small 
replica-symmetry breaking fluctuations. i— ir— i 

The full analytic results for this case can be found inEJ'EiJ. Here we sim- 
ply exhibit graphically the comparisons between theory and simulation for the 
Hopfield model for a = 0.1 and deterministic microdynamics. Fig 11 shows 
flows in (m,r), with time implicit, and it may be observed that the compari- 
son is quite good (but not perfect) ; it also clearly shows the need for (at least) 
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Figure 11: Macroscopic flow trajectories for a Hopfield model with storage capacity a = 0.1 
and deterministic microscopic dynamics (/3 = oo); dots indicate simulations {N = 32000), 
solid lines indicate analytic RS theory. The outer dashed line is the boundary predicted 
by RS theory; the inner dashed line indicates the onset of instability against RS-brealcing 
fluctuations, with stability on the side closer to the origin (from ^^). 



two order parameters. On the other hand, Fig 12, which shows the depen- 
dence of m and r on t, demonstrates that the theory misses a slowing-down 
effect seen in the simulations for non-retrieving situations (ie. ones in which 
hmf^oo 'Ti(t) = 0). One may further note that the slowing-down occurs before 
the system crosses the limit of replica-symmetry stability against small fluctua- 
tions, suggesting that its origin lies elsewhere than in the breakdown of the RS 
Ansatz used in the evaluation of D{z). Rather, one is driven to conclude that 
the problem lies in the loss of memory information inherent in the assumption 
of equipartitioning in the form used above. This implies that the set of ri must 
be expanded beyond just m and r, to include more microscopic effects. 



54 



■l-j 1 1-4-1- 



10 



8 10 



t 



t 



Figure 12: Temporal dependence of the order parameters for a Hopfield model with storage 
a = 0.1 and zero-temperature dynamics; dots indicate simulations {N = 32000), the other 
lines indicate RS theory shown with solid lines where stable, dashed lines where unstable. 
Time is measured in Monte Carlo steps per spin (from ^^). 



6.2 The sophisticated version of the theory: order function dynamics 

To improve on the theory as developed in the last section requires broadening 
the range of order parameters. Addition of a finite number of extra observables 
is not however expected to give more than just minor improvement; rather, a 
qualitative change of philosophy would seem to be required. To this end we 
propose instead for $7 the joint distribution of the spins and the fields modifying 
them, 

V{^, h;a)^^J2 ^^^-^^ [h~h,{a)] . (178) 

i 

The equation of motion for V can be obtained by a (more complicated) 
analogue of that used in the last section to study the evolution of to, r. Again 
this involves the use of self-averaging and equipartitioning to close the equa- 
tions, but now the latter is much less restrictive (only microstates with the 
same distribution V are taken to contribute equally). Details are given in 
E3. Here, in Fig. 13, we show only the quahty of the result used to calculate 
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Figure 13: Evolution of the binding energy of the Sherrington-Kirkpatrick spin glass (Jo = 0) 
from a random microscopic start. Comparison of simulations (A'^ = 8000, solid line) and 
predictions of the simple two-parameter (m, r) theory (RS stable, dotted; RS unstable, 
dashed) and of the advanced order-function theory of section V, for /3 = oo. Note that the 
two solid lines are almost coincident (from ^^). 

the time-dependence of the binding energy of the SK model; agreement with 
computer simulations is excellent. 

A detailed comparison of the two methods of allowing for macroscopic 
memory, the method of section 4 involving few two-time functions and the 
present method involving many single-time quantities 0, remains incomplete. 

7 Conclusion 

In these lectures I have tried to illustrate both the fundamental and the appli- 
cable character and interest of theoretical techniques developed to understand 
and quantify spin glasses. Discussion has been restricted to mean field theory 
or infinite-ranged systems, because of space restrictions. For further recent con- 
siderations of short-range spin glasses and other complementpjy experimental, 
simulational and theoretical studies the reader is referred toE3. 

References 
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